Device, system, and method of accessing storage

ABSTRACT

Device, system, and method of accessing storage. For example, a server includes: a Solid-State Drive (SSD) to store data; a memory mapper to map at least a portion of a storage space of the SSD into a memory space of the server; and a network adapter to receive a Small Computer System Interface (SCSI) read command incoming from a client device, to map one or more parameters of the SCSI read command into an area of the memory space of the server from which data is requested to be read by the client device, said area corresponding to a storage area of the SSD, and to issue a Remote Direct Memory Access (RDMA) write command to copy data directly to the client device from said area of the memory space corresponding to the SSD.

FIELD

Some embodiments are related to the field of electronic communication,and more particularly to accessing storage.

BACKGROUND

In some communication systems, a client device (e.g., a PersonalComputer) may communicate over a wired or wireless network with aserver. For example, the server may include a processor, a NetworkInterface Card (NIC), a short-term memory unit (e.g., Random AccessMemory (RAM)), and a long-term storage unit (e.g., a hard disk drive).

The server may receive from the client device a request to read datafrom the long-term storage unit. In response, the requested data iscopied (e.g., by the processor) from the long-term storage unit into theshort-term storage unit; and the NIC of the server then reads therequested data from the short-term memory unit and sends it over thenetwork to the client device.

Unfortunately, this process may result in a relatively slow flow ofdata. For example, the long-term memory unit may include mechanicallymoving parts, and may operate relatively slowly. Additionally, theserver may be required to serve multiple client devices, and incomingrequests may be placed into one or more queues, which need to be managedand handled. Furthermore, the requested data is copied into theshort-term memory unit of the server, thereby consuming time and/orprocessing power.

SUMMARY

Some embodiments include, for example, devices, systems, and methods ofaccessing storage.

In some embodiments, for example, a server includes: a Solid-State Drive(SSD) to store data; a memory mapper to map at least a portion of astorage space of the SSD into a memory space of the server; a networkadapter to receive a Small Computer System Interface (SCSI) read commandincoming from a client device, to map one or more parameters of the SCSIread command into an area of the memory space of the server from whichdata is requested to be read by the client device, said areacorresponding to a storage area of the SSD, and to issue a Remote DirectMemory Access (RDMA) write command to copy data directly to the clientdevice from said area of the memory space corresponding to the SSD.

In some embodiments, the network adapter is to map the one or moreparameters of the SCSI read command by calling an ApplicationProgramming Interface (API) of the SSD.

In some embodiments, the one or more parameters of the SCSI read commandinclude at least one parameter selected from the group consisting of: aLogical Unit (LUN) parameter, a sector parameter, and a size parameter.

In some embodiments, the network adapter is to perform the RDMA writecommand based on a Steering Tag provided by the client device.

In some embodiments, the server further includes a Random Access Memory(RAM) to temporarily store data, and the network adapter is to copy thedata to the client device from said area of the memory space of theserver corresponding to the SSD through a direct data path excluding theRAM.

In some embodiments, the server further includes a Central ProcessingUnit (CPU) to execute instructions, and the network adapter is to copythe data to the client device from said area of the memory space of theserver corresponding to the SSD using a direct copy process excludingsaid CPU.

In some embodiments, completion of the RDMA write command, the networkadapter is to issue to the client device a SCSI Command Response (CR) toindicate said completion.

In some embodiments, substantially immediately with issuing the RDMAwrite command, the network adapter is to post to the client device aSCSI Command Response (CR) associated with a fencing requirement.

In some embodiments, the network adapter includes a Direct DataPlacement (DDP) network adapter, and the DDP network adapter is to copythe data to the client device from said area of the memory space of theserver corresponding to the SSD through a DDP data path.

In some embodiments, a computer includes: a Solid-State Drive (SSD) tostore data; a memory mapper to map at least a portion of a storage spaceof the SSD into a memory space of the computer; and a network adapter toreceive a Small Computer System Interface (SCSI) write command incomingfrom a client device, to map one or more parameters of the SCSI writecommand into an area of the memory space of the computer to which datais requested to be written by the client device, said area correspondingto a storage area of the SSD, and to issue a Remote Direct Memory Access(RDMA) read command to copy data directly from the client device to saidarea of the memory space corresponding to the SSD.

In some embodiments, the network adapter is to map the one or moreparameters of the SCSI write command by calling an ApplicationProgramming Interface (API) of the SSD.

In some embodiments, the one or more parameters of the SCSI writecommand include at least one parameter selected from the groupconsisting of: a Logical Unit (LUN) parameter, a sector parameter, and asize parameter.

In some embodiments, the network adapter is to perform the RDMA readcommand based on a Steering Tag provided by the client device.

In some embodiments, the computer further includes a Random AccessMemory (RAM) to temporarily store data, and the network adapter is tocopy the data from the client device to said area of the memory space ofthe computer corresponding to the SSD through a direct data pathexcluding the RAM.

In some embodiments, the computer further includes a Central ProcessingUnit (CPU) to execute instructions, and the network adapter is to copythe data from the client device to said area of the memory space of thecomputer corresponding to the SSD using a direct copy process excludingsaid CPU.

In some embodiments, upon completion of the RDMA read command, thenetwork adapter is to issue to the client device a SCSI Command Response(CR) to indicate said completion.

In some embodiments, substantially immediately with issuing the RDMAread command, the network adapter is to post to the client device a SCSICommand Response (CR) associated with a fencing requirement.

In some embodiments, the network adapter includes a Direct DataPlacement (DDP) network adapter, and the DDP network adapter is to copythe data from the client device to said area of the memory space of thecomputer corresponding to the SSD through a DDP data path.

In some embodiments, a method includes: mapping at least a portion of astorage space of a Solid-State Drive (SSD) of a server into a memoryspace of the server; receiving, by a network adapter of the server, aSmall Computer System Interface (SCSI) read command incoming from aclient device; mapping one or more parameters of the SCSI read commandinto an area of the memory space of the server from which data isrequested to be read by the client device, said area corresponding to astorage area of the SSD; and issuing a Remote Direct Memory Access(RDMA) write command to copy data directly to the client device fromsaid area of the memory space corresponding to the SSD.

In some embodiments, mapping one or more parameters of the SCSI readcommand includes: calling an Application Programming Interface (API) ofthe SSD; and the one or more parameters of the SCSI read command includeat least one parameter selected from the group consisting of: a LogicalUnit (LUN) parameter, a sector parameter, and a size parameter.

In some embodiments, the method includes: copying the data to the clientdevice from said area of the memory space of the server corresponding tothe SSD through a direct data path which excludes a Random Access Memory(RAM) of the server and further excludes a Central Processing Unit (CPU)of the server.

In some embodiments, the method includes: upon completion of the RDMAwrite command, issuing to the client device a SCSI Command Response (CR)to indicate said completion.

In some embodiments, the method includes: substantially immediately withissuing the RDMA write command, posting to the client device a SCSICommand Response (CR) associated with a fencing requirement.

In some embodiments, the network adapter includes a Direct DataPlacement (DDP) network adapter, and the method includes: copying thedata to the client device from said area of the memory space of theserver corresponding to the SSD through a DDP data path.

In some embodiments, a method includes: mapping at least a portion of astorage space of a Solid-State Drive (SSD) of a server into a memoryspace of the server; receiving, by a network adapter of the server, aSmall Computer System Interface (SCSI) write command incoming from aclient device; mapping one or more parameters of the SCSI write commandinto an area of the memory space of the server to which data isrequested to be written by the client device, said area corresponding toa storage area of the SSD; and issuing a Remote Direct Memory Access(RDMA) read command to copy data directly from the client device to saidarea of the memory space corresponding to the SSD.

In some embodiments, mapping one or more parameters of the SCSI writecommand includes: calling an Application Programming Interface (API) ofthe SSD; and the one or more parameters of the SCSI write commandinclude at least one parameter selected from the group consisting of: aLogical Unit (LUN) parameter, a sector parameter, and a size parameter.

In some embodiments, the method includes: copying the data from theclient device to said area of the memory space of the servercorresponding to the SSD through a direct data path which excludes aRandom Access Memory (RAM) of the server and further excludes a CentralProcessing Unit (CPU) of the server.

In some embodiments, the method includes: upon completion of the RDMAread command, issuing to the client device a SCSI Command Response (CR)to indicate said completion.

In some embodiments, the method includes: substantially immediately withissuing the RDMA read command, posting to the client device a SCSICommand Response (CR) associated with a fencing requirement.

In some embodiments, the network adapter includes a Direct DataPlacement (DDP) network adapter, and the method includes: copying thedata from the client device to said area of the memory space of theserver corresponding to the SSD through a DDP data path.

Some embodiments may include, for example, a computer program productincluding a computer-useable medium including a computer-readableprogram, wherein the computer-readable program when executed on acomputer causes the computer to perform methods in accordance with someembodiments of the invention.

Some embodiments may provide other and/or additional benefits and/oradvantages.

BRIEF DESCRIPTION OF THE DRAWINGS

For simplicity and clarity of illustration, elements shown in thefigures have not necessarily been drawn to scale. For example, thedimensions of some of the elements may be exaggerated relative to otherelements for clarity of presentation. Furthermore, reference numeralsmay be repeated among the figures to indicate corresponding or analogouselements. The figures are listed below.

FIG. 1 is a schematic block diagram illustration of a system inaccordance with some demonstrative embodiments of the invention.

FIG. 2 is a schematic flow-chart of a method of reading from storage, inaccordance with some demonstrative embodiments of the invention.

FIG. 3 is a schematic flow-chart of a method of writing into storage, inaccordance with some demonstrative embodiments of the invention.

DETAILED DESCRIPTION

In the following detailed description, numerous specific details are setforth in order to provide a thorough understanding of some embodimentsof the invention. However, it will be understood by persons of ordinaryskill in the art that some embodiments may be practiced without thesespecific details. In other instances, well-known methods, procedures,components, units and/or circuits have not been described in detail soas not to obscure the discussion.

The terms “plurality” or “a plurality” as used herein include, forexample, “multiple” or “two or more”. For example, “a plurality ofitems” includes two or more items.

Although portions of the discussion herein relate, for demonstrativepurposes, to wired links and/or wired communications, some embodimentsare not limited in this regard, and may include one or more wired orwireless links, may utilize one or more components of wirelesscommunication, may utilize one or more methods or protocols of wirelesscommunication, or the like. Some embodiments may utilize wiredcommunication and/or wireless communication.

The terms “Remote Direct Memory Access” and “RDMA” as used hereininclude, for example, hardware and/or software and/or infrastructureand/or fabric and/or links and/or adapters and/or architectures, whichallow direct hardware access to write from a local memory to a remote orlocal node's memory and/or to read from a remote or local node's memoryto a local node's memory; for example, substantially without involvingthe Operating System (OS) of the remote computer, or by substantiallybypassing the OS of the remote computer. Additionally or alternatively,RDMA may be implemented, for example, substantially without involvingthe OS of the initiating node, or by substantially bypassing the OS ofthe initiating node. In some embodiments, RDMA may providehigh-throughput, low-latency, zero-copy networking; and may allow anetwork adapter (e.g., a Network Interface Card (NIC), a Host ChannelAdapter (HCA), or the like) to transfer data directly to or fromapplication memory, eliminating the need to copy data betweenapplication memory and OS data buffers; as well as eliminatingutilization of processors, caches, and/or context switches, and furtherallowing data transfer simultaneously and/or in parallel with otheroperations. In some embodiments, the term “RDMA” may include mechanismsor operations that are similar to Remote Direct Memory Access; or thatcan be used instead of or in addition to Remote Direct Memory Access,for example, iWarp or Direct Data Placement (DDP), which may be used inconjunction with various types of infrastructures, e.g., InfiniBand,Ethernet, or the like; or other offload mechanisms.

The terms “communication unit” or “Network Interface Card” or “NIC” or“NIC/HCA” as used herein include, for example, a Host Channel Adapter(HCA), a RDMA-capable NIC or HCA or HBA, an Ethernet NIC or HCA or HBA,a NIC or HCA or HBA or card or adapter with TCP offload capabilities, aHost Bus Adapter (HBA), a Fibre Channel (FC) HBA, a Fibre Channel overEthernet (FCoE) HBA, a RDMA-capable hardware component or card oradapter, a NIC or HCA or HBA that supports Direct Data Placement (e.g.,an Internet SCSI (iSCSI) HBA), a NIC or HCA or HBA having OS-bypasscapabilities, an InfiniBand NIC or HCA or card or adapter, an iWarp NICor HCA or card or adapter, a card or adapter able to bypass OS and/orkernel and/or driver(s), a card or adapter able to directly access amemory of a remote device or server or node, or the like.

The term “Ethernet” as used herein includes, for example, Ethernet,Ethernet in accordance with IEEE 802.3 standard and/or 802.2 standardand/or other standards, Gigabit Ethernet (GEth), 10-Gigabit Ethernet,100-Gigabit Ethernet, Fast Ethernet, Converged Ethernet, or other typesof Ethernet.

The terms “Operating System (OS) bypassing” or “OS bypass” as usedherein include, for example, a substantially complete or a partial OSbypassing, a substantially complete or a partial kernel bypassing, asubstantially complete or a partial bypass of a driver, or the like. Insome embodiments, OS bypass may be implemented by using system calls toOS kernel in order to execute connection control and/or memoryregistration for RDMA, while send and/or receive operations of data areperformed mostly or solely by OS bypass.

The terms “Solid-State Drive” or “SSD” as used herein includes, forexample, a data storage unit which utilizes or includes one or moresolid-state memory components to store data; a long-term solid-statestorage unit; a storage unit which includes flash memory and is able toemulate a hard disk drive; a long-term storage unit that includessubstantially no moving parts, or substantially no mechanically-movingparts; a long-term storage unit that exclusively includes onlynon-moving parts and/or non-mechanically-moving parts; a long-termstorage unit which does not require cooling and/or does not include afan; a substantially completely silent and/or noiseless long-termstorage unit; a long-term storage unit having a substantially constantand/or deterministic read performance, or having a substantiallyconstant and/or deterministic read time and/or seek time acrosssubstantially the entire storage space; a low power consumptionlong-term storage unit; a long-term storage unit having high mechanicalreliability, and/or ability to endure shocks and/or vibration and/orhigh altitude; a long-term storage unit having substantially nomechanical delays, having low access time and low latency; aFlash-memory based SSD; an array or an arrangement of two or more SSDs,of a common type or of different types; a Redundant Array of IndependentDisks (RAID) which includes two or more SSD; or the like.

The terms “Random Access Memory” or “RAM” as used herein include, forexample, one or more integrated circuits able to store data to beaccessed randomly or pseudo-randomly or in substantially any order; avolatile memory which requires power in order to maintain the storeddata; Static RAM (SRAM), or RAM which stores a bit of data using a stateof a flip-flop; Dynamic RAM (DRAM), or RAM which stores a bit of datausing a charge in a capacitor or a transistor gate; Synchronous DRAM(SDRAM); a memory unit having one or more RAM packages or RAM modules,for example, one or more Single In-line Memory Modules (SIMMs) or DualIn-line Memory Modules (DIMMs); a Double Data Rate (DDR) SDRAM; or thelike.

Some embodiments may be used in conjunction with various devices andsystems, for example, a Personal Computer (PC), a desktop computer, amobile computer, a laptop computer, a notebook computer, a tabletcomputer, a server computer, a handheld computer, a handheld device, aPersonal Digital Assistant (PDA) device, a handheld PDA device, anon-board device, an off-board device, a hybrid device (e.g., a deviceincorporating functionalities of multiple types of devices, for example,PDA functionality and cellular phone functionality), a vehicular device,a non-vehicular device, a mobile or portable device, a non-mobile ornon-portable device, a wireless communication station, a wirelesscommunication device, a wireless Access Point (AP), a wireless BaseStation (BS), a Mobile Subscriber Station (MSS), a wired or wirelessNetwork Interface Card (NIC), a wired or wireless router, a wired orwireless modem, a wired or wireless network, a Local Area Network (LAN),a Wireless LAN (WLAN), a Metropolitan Area Network (MAN), a Wireless MAN(WMAN), a Wide Area Network (WAN), a Wireless WAN (WWAN), a PersonalArea Network (PAN), a Wireless PAN (WPAN), devices and/or networksoperating in accordance with existing IEEE 802.11, 802.11a, 802.11b,802.11g, 802.11n, 802.16, 802.16d, 802.16e, 802.16m standards and/orfuture versions and/or derivatives of the above standards, units and/ordevices which are part of the above networks, one way and/or two-wayradio communication systems, cellular radio-telephone communicationsystems, a cellular telephone, a wireless telephone, a PersonalCommunication Systems (PCS) device, a PDA device which incorporates awireless communication device, a mobile or portable Global PositioningSystem (GPS) device, a device which incorporates a GPS receiver ortransceiver or chip, a device which incorporates an RFID element or tagor transponder, a device which utilizes Near-Field Communication (NFC),a Multiple Input Multiple Output (MIMO) transceiver or device, a SingleInput Multiple Output (SIMO) transceiver or device, a Multiple InputSingle Output (MISO) transceiver or device, a device having one or moreinternal antennas and/or external antennas, a “smartphone” device, awired or wireless handheld device (e.g., BlackBerry®, Palm® Treo™), aWireless Application Protocol (WAP) device, or the like.

Some embodiments may be used in conjunction with one or more types ofwireless communication signals and/or systems, for example, RadioFrequency (RF), Infra Red (IR), Frequency-Division Multiplexing (FDM),Orthogonal FDM (OFDM), OFDM Access (OFDMA), Time-Division Multiplexing(TDM), Time-Division Multiple Access (TDMA), Extended TDMA (E-TDMA),General Packet Radio Service (GPRS), extended GPRS, Code-DivisionMultiple Access (CDMA), Wideband CDMA (WCDMA), CDMA 2000, Multi-CarrierModulation (MDM), Discrete Multi-Tone (DMT), Bluetooth®, GlobalPositioning System (GPS), IEEE 802.11 (“Wi-Fi”), IEEE 802.16 (“Wi-Max”),ZigBee™, Ultra-Wideband (UWB), Global System for Mobile communication(GSM), 2G, 2.5G, 3G, Third Generation Partnership Project (3GPP), 3GPPLong Term Evolution (LTE), 3.5G, or the like. Some embodiments may beused in conjunction with various other devices, systems and/or networks.

Although some portions of the discussion herein relate, fordemonstrative purposes, to a fast or high-speed interconnectinfrastructure, to a fast or high-speed interconnect component oradapter with OS bypass capabilities, to a fast or high-speedinterconnect card or Network Interface Card (NIC) with OS bypasscapabilities, or to a to a fast or high-speed interconnectinfrastructure or fabric, some embodiments are not limited in thisregard, and may be used in conjunction with other infrastructures,fabrics, components, adapters, host channel adapters, cards or NICs,which may or may not necessarily be fast or high-speed or with OS bypasscapabilities. For example, some embodiments may be utilized inconjunction with InfiniBand (IB) infrastructures, fabrics, components,adapters, host channel adapters, cards or NICs; with iWarpinfrastructures, fabrics, components, adapters, host channel adapters,cards or NICs; with Ethernet infrastructures, fabrics, components,adapters, host channel adapters, cards or NICs; with Ethernet TCPoffload infrastructures, fabrics, components, adapters, host channeladapters, cards or NICs; with Ethernet (e.g., Fast Ethernet, GigabitEthernet (GEth), 10-Gigabit Ethernet, 100-Gigabit Ethernet, or othertypes of Ethernet) infrastructures, fabrics, components, adapters, hostchannel adapters, cards or NICs; with infrastructures, fabrics,components, adapters, host channel adapters, cards or NICs that have OSwith infrastructures, fabrics, components, adapters, host channeladapters, cards or NICs that allow a user mode application to directlyaccess such hardware and bypassing a call to the operating system(namely, with OS bypass capabilities); with infrastructures, fabrics,components, adapters, host channel adapters, cards or NICs that haveOS-bypass capabilities; with infrastructures, fabrics, components,adapters, host channel adapters, cards or NICs that are connectionlessand/or stateless; and/or other suitable hardware.

FIG. 1 schematically illustrates a block diagram of a system 100 inaccordance with some demonstrative embodiments. System 100 includes oneor more servers, for example, a server 110. System 100 further includesone or more client devices, for example, a client device 150. AlthoughFIG. 1 shows, for demonstrative purposes, one server 110, other numbersof servers may be used. Similarly, although FIG. 1 shows, fordemonstrative purposes, one client device 150, other numbers of clientdevices may be used. In some embodiments, client device 150 may operateas initiator, and server 110 may operate as target.

Server 110 includes, for example, a processor 111, an input unit 112, anoutput unit 113, a RAM 114, and a SSD 115. Server 110 may optionallyinclude other suitable hardware components and/or software components.Server 110 may be implemented, for example, using a computing platformor a server computer.

Processor 111 may include, for example, a Central Processing Unit (CPU),a Digital Signal Processor (DSP), one or more processor cores, amicroprocessor, a host processor, a controller, a plurality ofprocessors or controllers, a chip, a microchip, one or more circuits,circuitry, a logic unit, an Integrated Circuit (IC), anApplication-Specific IC (ASIC), or any other suitable multi-purpose orspecific processor or controller. Processor 111 may executeinstructions, for example, of an Operating System (OS) 117 of server 110or of one or more software applications 118.

Input unit 112 may include, for example, a keyboard, a keypad, a mouse,a touch-pad, a track-ball, a track-wheel, a thumb-wheel, a scroll-wheel,a stylus, one or more buttons or sliders, a microphone, or othersuitable pointing device or input device.

Output unit 113 may include, for example, a monitor, a screen, a CathodeRay Tube (CRT) display unit, a Liquid Crystal Display (LCD) displayunit, a plasma display unit, a projector, a projection device, atelevision, a High Definition Television (HDTV) display unit, one ormore audio speakers, or other suitable output devices.

The RAM 114 may include, for example, one or more DIMMs or SIMMs orother volatile memory components.

The SSD 115 may include, for example, one or more SSDs, or an array orarrangement of multiple SSDs. The SSD 115 may be connected to, forexample, a Small Computer System Interface (SCSI) interface 122, e.g., aSCSI bus, SCSI connector, SCSI controller, or SCSI card. The SSD 115 maybe controlled, for example, using a SSD Application ProgrammingInterface (API) 123. In some embodiments, a Advanced TechnologyAttachment (ATA) or Serial ATA (SATA) interfaces, bus, connectors,drives, cards and/or SSDs may be used, for example, instead of SCSI orin addition to SCSI.

Server 110 may further include a communication unit, for example, awired or wireless Network Interface Card (NIC), a Host Channel Adapter(HCA), an InfiniBand HCA, a wired or wireless modem, a wired or wirelessrouter or switch or hub, a wired or wireless receiver and/ortransmitter, a wired or wireless transmitter-receiver and/ortransceiver, a Radio Frequency (RF) communication unit or transceiver,or other units able to transmit and/or receive signals, blocks, frames,transmission streams, packets, messages and/or data. Optionally, thecommunication unit may include, or may be associated with, one or moreantennas, for example, a dipole antenna, a monopole antenna, anomni-directional antenna, an end fed antenna, a circularly polarizedantenna, a micro-strip antenna, a diversity antenna, or the like.

For demonstrative purposes, server 110 includes a NIC/HCA 119. Forexample, the NIC/HCA 119 may include a fast or high-speed interconnectcard or adapter or HCA; a NIC or HCA having OS bypass and/or RDMAcapabilities; an InfiniBand (IB) NIC or HCA; an Ethernet NIC or HCA; anEthernet (e.g., Fast Ethernet, Gigabit Ethernet (GEth), 10-GigabitEthernet, Converged Ethernet NIC (C-NIC), 100-Gigabit Ethernet, or othertypes of Ethernet) NIC or HCA; a NIC or HCA that allows an application(e.g., a user-mode application) to directly access hardware, and/or todirectly access remote hardware (e.g., using RDMA); a RDMA-capable NICor HCA; a NIC or HCA that allows a user-mode application to bypasscall(s) to a local OS and/or to an OS of a remote device; aconnectionless and/or stateless NIC or HCA; and/or other suitablehardware. Optionally, NIC 119 may be associated with a driver 121, forexample, a software module or an interface allowing other softwarecomponents of server 110 (e.g., the OS 117 or the applications 118) tointeract with the NIC 119.

In some embodiments, the components of server 110 may be arranged orenclosed in a common housing or packaging, and may be interconnected orcoupled or operably associated using one or more wired or wirelesslinks. In other embodiments, components of server 110 may be distributedamong multiple or separate devices or locations.

Client device 150 includes, for example, a processor 151, an input unit152, an output unit 153, a RAM 154 or other short-term memory unit, anoptional Hard Disk Drive (HDD) 155 or other long-term storage unit(although in some embodiments, client device 150 need not include along-term storage unit), an OS 157, one or more software applications158, and a NIC 159 optionally associated with a driver 161.

In some embodiments, for example, the NIC 119 of server 110 may beconnected to the NIC 159 of client device 150 through a network orthrough one or more wired and/or wireless links, for example, a link199. Link 199 may include, for example, a fast or high-speedinterconnect link; a link able to allow OS bypassing; an InfiniBand (IB)link; an Ethernet (e.g., Fast Ethernet, Gigabit Ethernet (GEth),10-Gigabit Ethernet, 100-Gigabit Ethernet, or other types of Ethernet)link; a Fibre Channel (FC) link; a link that allows a user-modeapplication of the client device 150 to directly access hardware (e.g.,RAM 114) of server 110; a link that allows a user-mode application ofthe client device 150 to utilize RDMA in order to directly access remotehardware (e.g., RAM 114) of server 110; a RDMA-capable link; a link thatallows Direct Data Placement (DDP); a link that allows a user-modeapplication to bypass call(s) to a local OS and/or to an OS of a remotedevice; a link that allows connectionless and/or statelesscommunication; and/or other suitable wired or wireless links, fabrics,or infrastructures.

In accordance with some embodiments, a software and/or hardwarearchitecture may be used to allow rapid and/or efficient access by theclient device 150 to the SSD 115 of server 110. For example, the clientdevice 150 issues a request (e.g., a read request or a write request),which is transferred from the client device 150 over the link 199 to theserver 110; and a response is transferred back from the server 110 overthe link 199 to the client device 150. In some embodiments,substantially the entire flow (namely, the incoming request, thehandling of the request, and the outgoing response) runs in a singlerunning context.

In accordance with some embodiments, the SSD 115 is mapped into thememory space (e.g., to the physical and/or virtual memory space) ofserver 110. The mapping is performed by a mapper module 124, which maybe implemented, for example, as part of SSD 115, as part of OS 117, as amemory file system, as part of application(s) 118, as part of SCSIinterface 122, as part of SCSI API 123 or utilizing the SSD API 123, oras a stand-alone or separate hardware component and/or softwarecomponent. The memory mapper 124 controls memory pages, and controlsmapping of memory pages to SSD 115.

In some embodiments, system 100 allows rapid and/or efficient readaccess by the client device 150 to the SSD 115 of server 110. Forexample, the NIC/HCA 119 of server 110 receives from the client device150 a SCSI read command. The SCSI read command includes one or moreparameters, for example, a Logical Unit (LUN) parameter, a sectorparameter, and a size parameter. The NIC/HCA 119 calls the SSD API 123in order to map the parameters of the SCSI read command into a physicaladdress of the SSD 115, namely, a pointer to a physical memory area fromwhich the requested data is to be read. The NIC/HCA 119 issues aRDMA-write (RW) in order to copy data directly from that physicaladdress of the SSD 115 to the client device 150. The RDMA-write isperformed, for example, based on a Steering-Tag (STag) or other networkVirtual Address (VA) indicator provided by the client device 150,indicating the memory location in client device 150 into which the datais to be placed. In some embodiments, once the RDMA-write operationcompletes, the NIC/HCA 119 sends a SCSI Command Response (CR) to theclient device 150, to indicate completion; the CR is sent serially,namely, once the RDMA-write operation completes. In other embodiments,the CR may be posted substantially immediately with the RDMA-write,using a fencing mechanism; for example, the CR is issued with theRDMA-write operation and together with a fencing requirement, whichensures that the CR is received at the client device 150 only uponcompletion of the RDMA-write.

In some embodiments, system 100 allows rapid and/or efficient writeaccess by the client device 150 to the SSD 115 of server 110. Forexample, the NIC/HCA 119 of server 110 receives from the client device150 a SCSI write command. The SCSI write command includes one or moreparameters, for example, a Logical Unit (LUN) parameter, a sectorparameter, and a size parameter. The NIC/HCA 119 calls the SSD API 123in order to map the parameters of the SCSI write command into a physicaladdress of the SSD 115, namely, a pointer to a physical memory area intowhich the requested data is to be written. The NIC/HCA 119 issues aRDMA-read (RR) in order to copy data directly from the client device 150to that physical address of the SSD 115. The RDMA-read is performed, forexample, based on a Steering-Tag (STag) or other network Virtual Address(VA) indicator provided by the client device 150, indicating the memorylocation in client device 150 from which the data is to be read. In someembodiments, once the RDMA-read operation completes, the NIC/HCA 119sends a SCSI Command Response (CR) to the client device 150, to indicatecompletion; the CR is sent serially, namely, once the RDMA-readoperation completes. In other embodiments, the CR may be postedsubstantially immediately with the RDMA-read, using a fencing mechanism;for example, the CR is issued with the RDMA-read operation and togetherwith a fencing requirement, which ensures that the CR is received at theclient device 150 only upon completion of the RDMA-read.

In some embodiments, a single-context command path 130 is used,including, for example, a hardware access library 131, a transport layer132, and a Storage Forwarding Engine (SFE) 133 (e.g., a SCSI forwardingengine). Optionally, a queue element may further be included, e.g., tomanage or handle queue(s) between the NIC/HCA 119 and the command path130. The transport layer 132 may be associated with the memory mapper124, which in turn controls the mapping of the SSD 115 storage spaceinto memory pages. A direct data path 136 allows rapid transfer of databetween the NIC/HCA 119 and the SSD 115, for example, using DMA or usingPeer to Peer or Point to Point (P2P) input/output. Other suitablearchitectures may be used.

In some embodiments, substantially the entire storage space of SSD 115is mapped at once to the physical memory space of server 110. In otherembodiments, for example, particularly if the storage space of SSD 115is significantly large, the storage space of SSD 115 is mapped to thephysical memory space in portions (e.g., “chunks” or “windows”); and theSSD API 123 may perform mapping of the relevant portion into memoryspace. In still other embodiments, for example, utilizing JournalingFlash File System (JFFS) or JFFS-2, one or more lookup operations may beused in order to retrieve the physical address in SSD 115 whichcorresponds to the parameters of an incoming SCSI request.

In some embodiments, system 100 may not support RDMA operations, but maystill allow rapid and/or efficient read/write access by the clientdevice 150 to the SSD 115 of server 110. For example, some embodimentsmay utilize Direct Data Placement (DDP) implemented using hardwareReady-to-Transmit (R2T) mechanism; or other suitable infrastructures,e.g., InfiniBand, iWarp, Gigabit Ethernet, FC, FCoE, iSCSI, or the like.In some embodiments, the transfer of data from client device 150 to SSD115, or vice versa, may be performed using OS-bypass, or by otherwisebypassing at least a portion of OS 117 and/or driver 121.

In some embodiments, the latency of serving an Input/Output (I/O)request is decreased, and the number of I/O operations per second isincreased, for example, to values which depend substantially exclusivelyon the capabilities of link 199 and SSD 115, and do not depend on thecapability and/or availability of processor 111, RAM 114 and/or OS 117.Some embodiments may be utilized in order to achieve efficiency of, forexample, under one microsecond I/O latency, and/or order-of-million I/Ooperations per seconds.

Some embodiments allow reduction of the cost of server 110. For example,in order to efficiently handle I/O operations, server 110 may notutilize the RAM 114, and therefore a smaller RAM may be included inserver 110. Similarly, in order to efficiently handle I/O operations,server 110 may not utilize processor 111, and therefore a slowerprocessor may be included in server 110. Similarly, a slower or cheapermemory bus 125 (e.g., which connects the processor 111 and the RAM 114)may be included in server 110. In some embodiments, thus, server 110 maybe implemented using slower or cheaper components, and may provideimproved I/O performance. Some embodiments may reduce the cost per I/Ooperation; may reduce the power consumption per I/O operation; mayreduce (e.g., to zero) the processor 111 processing cycles per I/Ooperation; may reduce (e.g., to substantially zero) the RAM 114utilization per I/O operation; may reduce (e.g., to substantially zero)the memory bus 125 utilization per I/O operation; or may otherwiseincrease system efficiency.

Some embodiments may utilize a memory-mapped SSD 115 in order to allowrapid access and/or “zero RAM copy” access to data stored in the SSD115. Some embodiments need not include, and need not utilize, a queue ormultiple queues (e.g., in a multi-layer architecture) between in orderto store, manage and/or handle incoming requests to access the datastored in the memory-mapped SSD 115. Some embodiments may avoid copyingof data into the RAM 114 (e.g., by the processor 111 or using a localDirect Memory Access (DMA) mechanism); and may avoid transfer of thedata from the SSD 115 to the NIC/HCA 119 through the RAM 114.

FIG. 2 is a schematic flow-chart of a method of reading from storage, inaccordance with some demonstrative embodiments of the invention.Operations of the method may be used, for example, by system 100 of FIG.1, by server 110 of FIG. 1, and/or by other suitable units, devicesand/or systems.

In some embodiments, the method may include, for example, mapping a SSDinto the physical memory space of a server (block 210).

In some embodiments, the method may include, for example, receiving by aNIC/HCA of the server, from a client device, a SCSI read command (block220).

In some embodiments, the method may include, for example, calling (e.g.,by the NIC/HCA of the server) a SSD API to map parameters of the SCSIread command into a physical address of the SSD (block 230).

In some embodiments, the method may include, for example, issuing (e.g.,by the NIC/HCA of the server) a RDMA-write in order to copy datadirectly from that physical address of the SSD to the client device(block 240).

In some embodiments, the method may include, for example, sending (e.g.,by the NIC/HCA of the server) a SCSI Command Response (CR) to the clientdevice, to indicate completion; optionally utilizing a fencing mechanism(block 250).

Other suitable operations or sets of operations may be used inaccordance with some embodiments.

FIG. 3 is a schematic flow-chart of a method of writing into storage, inaccordance with some demonstrative embodiments of the invention.Operations of the method may be used, for example, by system 100 of FIG.1, by server 110 of FIG. 1, and/or by other suitable units, devicesand/or systems.

In some embodiments, the method may include, for example, mapping a SSDinto the physical memory space of a server (block 310).

In some embodiments, the method may include, for example, receiving by aNIC/HCA of the server, from a client device, a SCSI write command (block320).

In some embodiments, the method may include, for example, calling (e.g.,by the NIC/HCA of the server) a SSD API to map parameters of the SCSIread command into a physical address of the SSD (block 330).

In some embodiments, the method may include, for example, issuing (e.g.,by the NIC/HCA of the server) a RDMA-read in order to copy data directlyfrom the client device into that physical address of the SSD (block340).

In some embodiments, the method may include, for example, sending (e.g.,by the NIC/HCA of the server) a SCSI Command Response (CR) to the clientdevice, to indicate completion; optionally utilizing a fencing mechanism(block 350).

Other suitable operations or sets of operations may be used inaccordance with some embodiments.

Discussions herein utilizing terms such as, for example, “processing,”“computing,” “calculating,” “determining,” “establishing”, “analyzing”,“checking”, or the like, may refer to operation(s) and/or process(es) ofa computer, a computing platform, a computing system, or otherelectronic computing device, that manipulate and/or transform datarepresented as physical (e.g., electronic) quantities within thecomputer's registers and/or memories into other data similarlyrepresented as physical quantities within the computer's registersand/or memories or other information storage medium that may storeinstructions to perform operations and/or processes.

Some embodiments may take the form of an entirely hardware embodiment,an entirely software embodiment, or an embodiment including bothhardware and software elements. Some embodiments may be implemented insoftware, which includes but is not limited to firmware, residentsoftware, microcode, or the like.

Furthermore, some embodiments may take the form of a computer programproduct accessible from a computer-usable or computer-readable mediumproviding program code for use by or in connection with a computer orany instruction execution system. For example, a computer-usable orcomputer-readable medium may be or may include any apparatus that cancontain, store, communicate, propagate, or transport the program for useby or in connection with the instruction execution system, apparatus, ordevice.

In some embodiments, the medium may be or may include an electronic,magnetic, optical, electromagnetic, InfraRed (IR), or semiconductorsystem (or apparatus or device) or a propagation medium. Somedemonstrative examples of a computer-readable medium may include asemiconductor or solid state memory, magnetic tape, a removable computerdiskette, a Random Access Memory (RAM), a Read-Only Memory (ROM), arigid magnetic disk, an optical disk, or the like. Some demonstrativeexamples of optical disks include Compact Disk-Read-Only Memory(CD-ROM), Compact Disk-Read/Write (CD-R/W), DVD, or the like.

In some embodiments, a data processing system suitable for storingand/or executing program code may include at least one processor coupleddirectly or indirectly to memory elements, for example, through a systembus. The memory elements may include, for example, local memory employedduring actual execution of the program code, bulk storage, and cachememories which may provide temporary storage of at least some programcode in order to reduce the number of times code must be retrieved frombulk storage during execution.

In some embodiments, input/output or I/O devices (including but notlimited to keyboards, displays, pointing devices, etc.) may be coupledto the system either directly or through intervening I/O controllers. Insome embodiments, network adapters may be coupled to the system toenable the data processing system to become coupled to other dataprocessing systems or remote printers or storage devices, for example,through intervening private or public networks. In some embodiments,modems, cable modems and Ethernet cards are demonstrative examples oftypes of network adapters. Other suitable components may be used.

Some embodiments may be implemented by software, by hardware, or by anycombination of software and/or hardware as may be suitable for specificapplications or in accordance with specific design requirements. Someembodiments may include units and/or sub-units, which may be separate ofeach other or combined together, in whole or in part, and may beimplemented using specific, multi-purpose or general processors orcontrollers. Some embodiments may include buffers, registers, stacks,storage units and/or memory units, for temporary or long-term storage ofdata or in order to facilitate the operation of particularimplementations.

Some embodiments may be implemented, for example, using amachine-readable medium or article which may store an instruction or aset of instructions that, if executed by a machine, cause the machine toperform a method and/or operations described herein. Such machine mayinclude, for example, any suitable processing platform, computingplatform, computing device, processing device, electronic device,electronic system, computing system, processing system, computer,processor, or the like, and may be implemented using any suitablecombination of hardware and/or software. The machine-readable medium orarticle may include, for example, any suitable type of memory unit,memory device, memory article, memory medium, storage device, storagearticle, storage medium and/or storage unit; for example, memory,removable or non-removable media, erasable or non-erasable media,writeable or re-writeable media, digital or analog media, hard diskdrive, floppy disk, Compact Disk Read Only Memory (CD-ROM), Compact DiskRecordable (CD-R), Compact Disk Re-Writeable (CD-RW), optical disk,magnetic media, various types of Digital Versatile Disks (DVDs), a tape,a cassette, or the like. The instructions may include any suitable typeof code, for example, source code, compiled code, interpreted code,executable code, static code, dynamic code, or the like, and may beimplemented using any suitable high-level, low-level, object-oriented,visual, compiled and/or interpreted programming language, e.g., C, C++,Java, BASIC, Pascal, Fortran, Cobol, assembly language, machine code, orthe like.

Functions, operations, components and/or features described herein withreference to one or more embodiments, may be combined with, or may beutilized in combination with, one or more other functions, operations,components and/or features described herein with reference to one ormore other embodiments, or vice versa.

While certain features of some embodiments have been illustrated anddescribed herein, many modifications, substitutions, changes, andequivalents may occur to those skilled in the art. Accordingly, thefollowing claims are intended to cover all such modifications,substitutions, changes, and equivalents.

1. A server comprising: a Solid-State Drive (SSD) to store data; amemory mapper to map at least a portion of a storage space of the SSDinto a memory space of the server; and a network adapter to receive aSmall Computer System Interface (SCSI) read command incoming from aclient device, to map one or more parameters of the SCSI read commandinto an area of the memory space of the server from which data isrequested to be read by the client device, said area corresponding to astorage area of the SSD, and to issue a Remote Direct Memory Access(RDMA) write command to copy data directly to the client device fromsaid area of the memory space corresponding to the SSD, wherein thenetwork adapter is to map the one or more parameters of the SCSI readcommand by calling an Application Programming Interface (API) of theSSD, and wherein the one or more parameters of the SCSI read commandcomprise at least one parameter selected from the group consisting of: aLogical Unit (LUN) parameter, a sector parameter, and a size parameter.2. The server of claim 1, wherein the network adapter is to perform theRDMA write command based on a Steering Tag provided by the clientdevice.
 3. The server of claim 1, wherein the server further comprises aRandom Access Memory (RAM) to temporarily store data, and wherein thenetwork adapter is to copy the data to the client device from said areaof the memory space of the server corresponding to the SSD through adirect data path excluding the RAM.
 4. The server of claim 1, whereinthe server further comprises a Central Processing Unit (CPU) to executeinstructions, and wherein the network adapter is to copy the data to theclient device from said area of the memory space of the servercorresponding to the SSD using a direct copy process excluding said CPU.5. The server of claim 1, wherein upon completion of the RDMA writecommand, the network adapter is to issue to the client device a SCSICommand Response (CR) to indicate said completion.
 6. The server ofclaim 1, wherein substantially immediately with issuing the RDMA writecommand, the network adapter is to post to the client device a SCSICommand Response (CR) associated with a fencing requirement.
 7. Theserver of claim 1, wherein the network adapter comprises a Direct DataPlacement (DDP) network adapter, and wherein the DDP network adapter isto copy the data to the client device from said area of the memory spaceof the server corresponding to the SSD through a DDP data path.
 8. Acomputer comprising: a Solid-State Drive (SSD) to store data; a memorymapper to map at least a portion of a storage space of the SSD into amemory space of the computer; and a network adapter to receive a SmallComputer System Interface (SCSI) write command incoming from a clientdevice, to map one or more parameters of the SCSI write command into anarea of the memory space of the computer to which data is requested tobe written by the client device, said area corresponding to a storagearea of the SSD, and to issue a Remote Direct Memory Access (RDMA) readcommand to copy data directly from the client device to said area of thememory space corresponding to the SSD, wherein the network adapter is tomap the one or more parameters of the SCSI write command by calling anApplication Programming Interface (API) of the SSD, and wherein the oneor more parameters of the SCSI write command comprise at least oneparameter selected from the group consisting of: a Logical Unit (LUN)parameter, a sector parameter, and a size parameter.
 9. The computer ofclaim 8, wherein the network adapter is to perform the RDMA read commandbased on a Steering Tag provided by the client device.
 10. The computerof claim 8, wherein the computer further comprises a Random AccessMemory (RAM) to temporarily store data, and wherein the network adapteris to copy the data from the client device to said area of the memoryspace of the computer corresponding to the SSD through a direct datapath excluding the RAM.
 11. The computer of claim 8, wherein thecomputer further comprises a Central Processing Unit (CPU) to executeinstructions, and wherein the network adapter is to copy the data fromthe client device to said area of the memory space of the computercorresponding to the SSD using a direct copy process excluding said CPU.12. The computer of claim 8, wherein upon completion of the RDMA readcommand, the network adapter is to issue to the client device a SCSICommand Response (CR) to indicate said completion.
 13. The computer ofclaim 8, wherein substantially immediately with issuing the RDMA readcommand, the network adapter is to post to the client device a SCSICommand Response (CR) associated with a fencing requirement.
 14. Thecomputer of claim 8, wherein the network adapter comprises a Direct DataPlacement (DDP) network adapter, and wherein the DDP network adapter isto copy the data from the client device to said area of the memory spaceof the computer corresponding to the SSD through a DDP data path.
 15. Amethod comprising: mapping at least a portion of a storage space of aSolid-State Drive (SSD) of a server into a memory space of the server;receiving, by a network adapter of the server, a Small Computer SystemInterface (SCSI) read command incoming from a client device; mapping oneor more parameters of the SCSI read command into an area of the memoryspace of the server from which data is requested to be read by theclient device, said area corresponding to a storage area of the SSD; andissuing a Remote Direct Memory Access (RDMA) write command to copy datadirectly to the client device from said area of the memory spacecorresponding to the SSD, wherein mapping one or more parameters of theSCSI read command comprises calling an Application Programming Interface(API) of the SSD, and wherein the one or more parameters of the SCSIread command comprise at least one parameter selected from the groupconsisting of: a Logical Unit (LUN) parameter, a sector parameter, and asize parameter.
 16. The method of claim 15, comprising: copying the datato the client device from said area of the memory space of the servercorresponding to the SSD through a direct data path which excludes aRandom Access Memory (RAM) of the server and further excludes a CentralProcessing Unit (CPU) of the server.
 17. The method of claim 15,comprising: upon completion of the RDMA write command, issuing to theclient device a SCSI Command Response (CR) to indicate said completion.18. The method of claim 15, comprising: substantially immediately withissuing the RDMA write command, posting to the client device a SCSICommand Response (CR) associated with a fencing requirement.
 19. Themethod of claim 15, wherein the network adapter comprises a Direct DataPlacement (DDP) network adapter, and wherein the method comprises:copying the data to the client device from said area of the memory spaceof the server corresponding to the SSD through a DDP data path.
 20. Amethod comprising: mapping at least a portion of a storage space of aSolid-State Drive (SSD) of a server into a memory space of the server;receiving, by a network adapter of the server, a Small Computer SystemInterface (SCSI) write command incoming from a client device; mappingone or more parameters of the SCSI write command into an area of thememory space of the server to which data is requested to be written bythe client device, said area corresponding to a storage area of the SSD;and issuing a Remote Direct Memory Access (RDMA) read command to copydata directly from the client device to said area of the memory spacecorresponding to the SSD, wherein mapping one or more parameters of theSCSI write command comprises calling an Application ProgrammingInterface (API) of the SSD, and wherein the one or more parameters ofthe SCSI write command comprise at least one parameter selected from thegroup consisting of: a Logical Unit (LUN) parameter, a sector parameter,and a size parameter.
 21. The method of claim 20, comprising: copyingthe data from the client device to said area of the memory space of theserver corresponding to the SSD through a direct data path whichexcludes a Random Access Memory (RAM) of the server and further excludesa Central Processing Unit (CPU) of the server.
 22. The method of claim20, comprising: upon completion of the RDMA read command, issuing to theclient device a SCSI Command Response (CR) to indicate said completion.23. The method of claim 20, comprising: substantially immediately withissuing the RDMA read command, posting to the client device a SCSICommand Response (CR) associated with a fencing requirement.
 24. Themethod of claim 20, wherein the network adapter comprises a Direct DataPlacement (DDP) network adapter, and wherein the method comprises:copying the data from the client device to said area of the memory spaceof the server corresponding to the SSD through a DDP data path.